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Abstract. We consider 3-parametric polynomials P*(x;q,t,s) which replace the 
A n -series interpolation Macdonald polynomials P*(x;q,t) for the £>C n -type root 
system. For these polynomials we prove an integral representation, a combinato- 
rial formula, Pieri rules, Cauchy identity, and we also show that they do not satisfy 
any rational (/-difference equation. As s — > oo the polynomials P*(x;q,t,s) become 
P*(x;q,t). We also prove a binomial formula for 6-parametric Koornwinder polyno- 
mials. 
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0. Introduction 

The higher Capelli identities [N,Okl-2] describe two faces of a distinguished 
linear basis in the center of i/($j[(n)). The elements of this basis, which are called 
quantum immanants, have also a third and a very important face. It is their 
Harish-Chandra image called shifted Schur functions, or s*-functions for short. The 
s*-functions have remarkably many applications, see [OO] and, for example, also 
[KOO]. In particular, they were used in the proof of the higher Capelli identities 
given in [Okl]. 

Since the definition of the s*-functions involves only the Weyl group and the 
half-sum of positive roots p, they can be generalized in a quite far reaching way. 
This was done in the A n case by F. Knop, G. Olshanski, S. Sahi, and the author in 
[KS,Kn,S,Ok3,003] and other papers, see References. This theory is remarkably 
parallel to the theory of ordinary Macdonald polynomials P tl (x;q : t) for the root 
system A n . In some aspects, such as e.g. duality [Ok3], §6, or binomial formula 
[Ok4], it looks even more natural. 
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The definition of the polynomials P*(x; q, t), which generalize shifted Schur func- 
tions in the way that 

S.AX) = lim — r-j— j- , 

where 

q x = (q Xl , .. ., q Xn ) 

and |// 1 stands for the number of squares in fx, is the following. By definition, the 
polynomial P*(x;q, t) is the unique, up to a scalar, polynomial of degree that 
is symmetric in variables 



(0.1) x l t n - 1 ,x 2 t n - 2 ,...,x n , 
and vanishes at the points 

P;(q\q,t) = 0, A, 

where A = (Ai, . . . , A n ) is a partition. We call these polynomials interpolation Mac- 
donald polynomials. They are also known as shifted or inhomogeneous Macdonald 
polynomials. 

The polynomials P*(x;q, t) are the Harish-Chandra image 

g Harish-Chandra isomorphism ^ f) ^ _ ^(n) 

of some distinguished linear basis of the commutative algebra D of Macdonald 
(/-difference operators. Recall that the algebra D acts diagonally in the basis of 
Macdonald polynomials and the Harish-Chandra homomorphism associates to any 
element of D its eigenvalues on the Macdonald polynomials P^(x; q, t) viewed as a 
function of q^. 

Informally, by analogy with [Okl], one can think of the polynomials P*(x; q, t) as 
of the Harish-Chandra image of certain Capelli-type Laplace operators on on some 
fictitious "quantum symmetric space" with an A n -type restricted root system. In 
the three cases 

t = q 9 , 0=^,1,2 

there exist certain quantum symmetric spaces (see [No2]) which are (/-analogs of 

U(n)/0(n), = 1/2, 

U(n) , 9 = 1, 

U(2n)/Sp(2n) , 9 = 2. 

Remark, that from the definition it is not obvious at all that the polynomials 
P^(x; q, t) are anyhow related to ordinary Macdonald polynomials. There is, there- 
fore, a natural question if any such relation exists for other root systems. The 
root system of maximal interest for applications is the non-reduced root system of 
type BC n , where we have even more "live" symmetric spaces than in the A n case. 
In [002], we considered the analogs of shifted Schur functions for other classical 
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In this paper we study the following natural BC n -type analogs of the interpola- 
tion Macdonald polynomials P*(x;q,t). By definition, the polynomial 

P;(x;q,t, S )eC(q,t,s)[xf 1 ,...,x± 1 } 

is the unique, up to a scalar, polynomial of degree which is symmetric in variables 
(0.1) and also invariant under the transformations 

Xit n ~ l S 

such that 

P;(q X ;q,t,s)=0 if n<£\ 
for any partition A. The vector p = (pi, . . . , p n ) such that 

qf> = (t n - 1 s,...,ts,s) 

plays the role of the half-sum of positive roots taken with multiplicities. 

Our analysis shows that this SC n -type case has some new and unexpected fea- 
tures. 

We prove that the polynomials P^(x; q, t, s) are actually very close to the polyno- 
mials P£(x; q, t). In particular, as s — > oo all negative powers of variables disappear 
and 

(0.2) P;(x;q,t, 8 )^P;(x;q,t), s - oo . 

Moreover, the new parameter s can be inserted into the two explicit formulas for 
P*(x; q, t) obtained in [Ok3], namely, the integral representation and the combinato- 
rial formula, in such a way that (0.2) becomes evident. This is done in sections 3-5, 
where the proofs are suitable modifications of the proofs from [Ok3] . There is, how- 
ever, also a major difference between the polynomials P*(x; q, t, s) and P*(x; q, t), 
see below. 

Recall that in the s*-functions case the combinatorial formula and the integral 
representation, which was called in that context the coherence property, have a clear 
interpretation in terms of the higher Capelli identities. The combinatorial formula 
reflects the formula (3.24) in [Okl] for the LHS of the higher Capelli identities, see 
[Okl], §3.7. The coherence property describes the action of the averaging operator 

center of U(gl(n)) — avemgmg > center of U(gl(N)) , n < N 

in the basis of quantum immanants. This action can be easily computed either 
by explicit averaging of the differential operator in the RHS of the higher Capelli 
identities, see [Okl], §5.1, or by using the vanishing property, see [OO], §10. 

The integral representation (3.1) is in fact a natural analog of the classical Weyl 
character formula. The denominator D(x*;q, t) in (3.1) is a straightforward gener- 
alization of the product in the denominator of the Weyl formula. The alternating 
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is, by definition, alternating sum of contributions of lower and upper bounds of in- 
tegration. One proves that the numerator is divisible by the denominator using, 
essentially, the fact that any (/-integral 

Z d q Z , I 7^ — 1 , 

is divisible by u — v in C[-u ±:L , v^ 1 ], see Appendix 1. 

On the other hand, the relation of the polynomials P^(x; q, t, s) to the Macdonald 
polynomials for the BC n root system is only an indirect one, via the binomial 
theorem. In fact, it is natural to work with Koornwinder polynomials [K], which 
generalize Macdonald polynomials for classical root systems and have as many 
as 6 parameters. We show that using the polynomials P*(x;q,t,s) one obtains 
a binomial formula for Koornwinder polynomials just along the same lines as in 
[Ok4]. The only property of the polynomials P*(x;q,t,s) needed in the proof of 
the binomial formula, as an abstract identity of symmetric polynomials, is the 
identification (4.1) of the top homogeneous term of P*(x; q, t, s). 

As an application of this binomial formula, or, more precisely, of its classical 
limit, one obtains a BC analog of the results of [004]. The details will be given in 
a forthcoming paper by G. Olshanski and the author. 

We conclude this introduction with remarks on some open problems and also 
some no go results. 

By the binomial theorem, see [003], §6, and [Ok4], the integral representation 
of P*(x;q,t) reflects the branching rule for the ordinary A n -tjpe Macdonald poly- 
nomials. It is likely that the integral representation of the polynomials P£(x; q, t, s) 
reflects branching rules for the Koornwinder polynomials. The 

s i-> st 1/2 

shift in the integral representation (3.1) suggests that the branching rules for Koorn- 
winder polynomials must have two natural steps just like the branching rules for 
characters of orthogonal and symplectic series. 

The combinatorial formula for the A n -type polynomials P*(x;q,t) and the in- 
tegral representation of P*(x;q,t) are equivalent to each other by virtue of the 
following symmetry, see [Ok3], formula (2.1), 

(Q 3) P;(aQ- A ",...,ag- Al ;Q,t) = Pjf(aq-"", ■ ■ ■ , ag-" 1 ; g, t) 

P*(a,...,a;q,t) P£(a, . . . ,a; q,t) 

between the argument x and the label \i of P^(x; q, t). Here a is an arbitrary num- 
ber. As a — > oo the symmetry (0.3) becomes the symmetry for ordinary Macdonald 
polynomials. 

It looks like no such symmetry connects the integral representation (3.1) to the 
combinatorial formula (5.3) for the SC n -type interpolation polynomials. First of 
all, the symmetry (0.3) uses the automorphism of the weight lattice 

A ^ -wo (A) , 
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which is trivial in the BC n case. Here w stands for the longest element of the 
Weyl group. Another obstruction to symmetry is that the weight function i^x/n in 
the combinatorial formula, which is the same ^x/n as in [M], VI. 6. 24 and Example 
VI.6.2.(b), 



(gA*-A|ig-«+% ( q >«-^ + Hi- i + 1 ) oc 

[UA) Y\/fi 11 ( ( Ai _ A . + 1 + 1 _^ X 

is, unlike the weight function in the integral representation (3.1), not invariant 
under transformations of the form 

q X it n-i s 

This problem is closely related to the following important difference between the 
polynomials A n and BC n case. The polynomials P*(x;q,t,s) do not satisfy any 
(/-difference equations with rational coefficients, see Appendix 2. Recall that there 
are very useful (/-difference equations for P*(x;q,t) discovered in [K,S2]. Explicit 
formulas for higher order difference equations can be found in [Ok4], §3. It seems to 
be a very interesting and important problem to find a geometric or representation- 
theoretic meaning of the Knop-Sahi difference equation and to understand why 
they do not have obvious SC-analogs. 

Finally, it is natural to ask if one can add some more parameters to the 3- 
parametric polynomials P*(x; q,t, s) and still preserve some of their properties. It 
has been shown in [Ok5] that the property 

P;{q X ;q,t,s) =0, fi<£ A, 

which requires the polynomials P*(x;q,t, s) to vanish at an infinite set of points 
q x , n (£ A, characterizes the polynomials P*(x; q,t, s) inside a very general class 
of interpolation polynomials. It follows from this characterization, see [Ok5], that 
neither integral representation nor combinatorial formula exist for any more general 
interpolation polynomials in that class. 

1. Definition and normalization 
Let (/, t, s be three parameters. We shall assume that 

qH j s k ^l for all iJ,keZ + . 

Set 

k = C(q,t,s) . 

We shall assume that 

(/, t G C and \q\, \t\ < 1 

wherever convergence is involved. We shall also need the square roots q 1/2 and t 1/2 . 
Consider the SC-type Weyl group 
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and its standard action on k[xf 1 , . . . , x^ 1 ]. Denote by 

A^Ckfxf 1 ,...,^ 1 ] 
the subalgebra of polynomials that are ^-invariant in new variables 
(1.1) x*=Xjt n_i s, i = l,...,n. 

Note that for any / G A tjS 

deg/>0 

and deg / = only for constant polynomials. Here deg / denotes the usual degree, 
that is the maximal total degree in x±, . . . , x n of all monomials of /. 

Definition 1.1. Let fx be a partition with at most n parts. By definition, 

(x±, . . . , x n \ q, t, s) 

is the element of A tjS satisfying the following conditions: 

(1) deg P*(x;q,t,s) < \fj,\, 

(2) P;(q x ;q,t,s) = if ia<£ A, 

(3) P;(q»;q,t,s) = H(ia,n;q,t,s). 

Here H (fi, n; q, t, s) is just a nonzero normalization constant which we shall spec- 
ify below. 

It is clear that existence of P* implies uniqueness. Below we shall give two 
explicit formulas for P* It is also clear that 

Proposition 1.1. The polynomials P*(x;q,t,s), where \i ranges over partitions 
with at most n parts, form a k-basis of the vector space A tjS . The degree 

(V) deg P*(x;q,t,s) = 

is exactly \n\. 

For any polynomial 

fix) E A t , s 

the coefficients / M in the expansion 

f{x) = Y,UPZ(x;q,t, a ) 

A* 

can be found from the following non-degenerate triangular (with respect to the 
ordering of partitions by inclusion) system of linear equations 

/(? a ) = £/ m p;(? a ;<z,M). 

At 

Here A ranges over all partitions with at most n parts. 

Now we shall specify the constant H{p>, n; q, £, s). Consider the diagram 



_ /. 



^ 
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as the following skew diagram 



H = M/n , 



where 



M = (2//i, fli+fl2,...,Hl + fi n , A»i - A» n , • • • , A»i - A»2, 0) , 
= (//i, /ii, . . . , fl X , fl X - fi n , . . . , fix - fi 2 , 0) . 



For the diagram 



(4,2,1,1) 



the diagram M looks as follows (the black squares correspond to the subdiagram 

■ □ □ □ □ 

■ □ □ 

■ □ 

■ □ 



Each square 



□ e ft 



has its mirror image Kl in the diagram fi. If □ is the i-th row and j-th column of 
fi then Kl is in the i-th row and (fii — j + l)-st column of fi. The mirror images of 
four squares □ G fi = (4, 2, 1, 1) are given in the following picture 



□ 



□ 



□ 
□ 



Recall the following notations of Macdonald, see [M] , Ch. I. Given a partition fi 



set 



n(fi) = Y / (i-l)»i = J2»'M- 1 )/ 2 



Recall that for each square □ = € fi the numbers 

a(D)=f H -j, a'(0)=j-l, 
l{U) = n' j -i, l'(n)=i-l, 

are called arm-length, arm-colength, legdength, and leg-colength respectively. The 
numbers 

a(B) = Ati + a'(D) , = 1(D) + 2(n - ^) 

are the arm-length and leg-length of the mirror image M of □ measured with respect 
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Definition 1.2. H(n, n; q, t, s) is the following normalization constant 

j.n(n) — 2(n— l)\fj,\ , x , N 

L__ JJ (? .<n> +V <n> _ ! .yC^ _ ! . 



Observe that 



Therefore, 



E a(®) = H + 3n(//) , 

= 2(n- l)|/x| -3n(/x). 



^.«;«.^)-|Sn(«° <D>+i * ,(D) -i) 



s — > oo 



which is the normalization constant for two-parameter interpolation Macdonald 
polynomials, see [Ok3]. 
Recall also that 

E a (°) = n tf) > E '( D ) = • 

Remark that P*(x;q,t,s) depends only on s 2 , not s, because the polynomial 
t, — s) satisfies all conditions of the definition of P£(x; q, t, s). 

2. Elementary properties 
We have the three following elementary propositions. 
Proposition 2.1. Suppose \x n > and put 

fl - 1 = (fix - 1, . . .,Hn ~ 1) . 

XTien 

(2.1) P;(x;g,t,a) = 

i \ 1 / 

Proof. It is clear the RHS of (2.1) satisfies all conditions of the definition 1.1 except 
for normalization. Evaluate it at x = q^. We obtain 

-M + |A*-I|£-3n(n-l)/2 s -2n x 

II (9^*""' - l ) (« 2 * n ~V* - 1) # U - 1. 9, f , *«) , 

i 

which equals H (/x, m; i, s) because 

= n(/i — 1) + n(n — l)/2 , 
n(//) = n(( / u - !)') + ]/x - 1| , 
\n\ = — 1| + n . 
This concludes the proof. □ 
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Proposition 2.2. Suppose fJL n = 0. Then 

(2.2) P*(x 1 ,..., x n _i, 1; q, t, s) = P*(x 1 , . . . , x n _i; (/, t, st) . 

Proposition 2.3. 

P M *(1M, • • • , l/x B ; = s^t^-^P^ • • • , x n ; q, t, s) . 

Proof. Again, it is clear that the LHS satisfies all conditions of the definition of 
P*(x; q,t, s) except for normalization. Compute H(fi,n; l/q,l/t,l/s). We obtain 

!__£_ JJ ^-.C-^^C, _ ^ ( s - 2g -«( B , t -,( B , _ !) _ 
which equals 

= s 2|A.| t (2»-2)| A .| fr ^ 7l .^ t)S ^ n 

3. Integral representation 

In this section we obtain a (/-integral representation of P*(x;q,t, s), which is a 
way to obtain the polynomial 

(x\ , . . . , x n ] (/, t, s) 
in n variables from the polynomial 

P^(yi,---,y n -i;q,t,st 1/2 ) 

in n — 1 variable. 

Basic facts about (/-integrals are recalled in the Appendix. Introduce some no- 
tations; set 

(a) oo = (1 - a)(l - ga)(l - <? 2 a) .... 
Consider the following products 

V(x u ...,x n )= det [(xr J+1 ~ x; {n - J+1) J 



U(x 1 ,...,x n ;y 1 ,...,y n - 1 ;q,t) = J| 



, (^( ;Ei ) ±1 (%) ±1 )o 



-^(^i, . . . , x n ; (/, t) = I I ((Xj + x i ) — [Xj + x • )) /,/ \±T7Z \±i\ 

f^j {t{ x i) { x j) )c 



Set also 



y*=y i t n - i - 1/2 s, z=l,...,n-l 
We shall integrate over the domain 



I d q y = d q y v ■ • / dqyn-x 

J VAX J XO J Xn 



' y-<x J X2 J x T . 

with respect to the following beta-type measure 

d q yi d q y n -i 



P q ,t(d q y) = V(y*;q,t)Il(x*;y*;q,t) 



Vl Vn-1 
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Theorem 3.1 (Integral representation). Suppose fi n = 0. Then 

(3.!) p;(,;,, M ) = ^^^.j ^MM^fe.un. 



Here 



C(/i, n) = 9 -(- 1 )/ 4 J] S ff (^ + ( n - 0) 



n(n-l)/4/ q _ „\(n-l) TT (q)oo (g^"~* +1 )oo 



where t = q and B q is the q-beta function (A. 3). 

This theorem together with proposition 2.1 gives a recursive formula for all 3- 
parametric interpolation Macdonald polynomials. 

Proposition 3.2. The RHS of (3.1) is an element of A t;S o/ degree < 

In the proof it will be convenient to consider the following analogs of the algebra 
A M - By 

Agckbf 1 ,...,^ 1 ] 

denote the subalgebra of polynomials that are (anti)-invariant with respect to per- 
mutation of x* and (anti)-invariant with respect to transformations 

* / *\ — l 

In particular, A|+ = A t s . We shall also write 

Ag[*] 

to stress the dependence on variables xi, . . . , x n . 

Proof of proposition 3.2. By analytic continuation we can assume that 

t = q\ 6=2k + l, k = 0,1,2, ... . 

In this case H(x; y; q, t) and D(x; q, t) belong to k^ 1 , . . . , x^ 1 ]. 
Note that 

U(x*;y*;q,t) = 

if 

Vi Q%ii Q %ii • • • i tq Xi 

or if 

yi = q~ x x i+1 , q~ 2 x i+1 , . . . , qt^Xi+x . 
Therefore, for any z, we can change the limits of integration in (3.1) as follows 

(3.2) P 1 d qVi ( ...) = f 1 l d qyi (...), r,s = l,...,2k. 
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In particular, we can replace our integration by the following 

r / \ r(qt) 1/2 xi y(gt) 1/2 x„_i / \ 

Jy<x \ J J{q/tyPx 2 J(q/ty/ 2 x n \ J 

Denote by f(x,y) the polynomial 

f(x, y) = V(y*; q, t) U(x*; y*; q, t) P;(y; q, t, st 1/2 ) . 

We have 

f(x,y) G A t>a [x], f(x,y) G A~ tVa [y] . 

In other words, f(x,y) is a VF-antiinvariant polynomial in variables y* with coeffi- 
cients in At jS [x]. 

The following determinants T>i form a linear basis in A~~ tl/2 [y] 



£>z(y) = det 



V 



*\-L 



\ 



J 1 



<i,j<n— 1 



where 



/i > / 2 > • • • > l ni > 
Using (A. 2) we can evaluate the integral 

*(qt)V 2 x 1 r (qt) 1 ^ 2 x n _ 1 



d q y 



Vi(y) 

(q/t)V2 X2 J(q/t)Wx n y 



explicitly and obtain, up to a constant factor, 

. . . (x*y* + «)-^ - ix* i+1 ) 1 * - (x* i+1 )- 1 * 



det 



1 1 



<i,j<n — 1 



This (n — 1) x (n — 1) determinant is equal to the following n x n determinant 



( {xt) h + {x\) 



det 



*\-h 



* \— /l 



Note that the result is an element of A t + [x]. 
Denote by / the integral 



/ P„ t (d q y)P*(y;q,t,sen 
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By the above considerations, we have 

Since (A.l) is always divisible by (u — v) it follows from (3.2) that I is divisible, for 
example, by 

(qt~ 1 xl - xl) . . . (q~ 1 x* 1 - x* 2 )(x\ - x* 2 )(qx\ - x 2 ) . . . (tq~ 1 xl - x 2 ) . 
By symmetry, / is divisible by D(x*; q,t), and moreover 

eA tta [x]. 



D(x*;q,t) 
Finally, observe that 

degV(y*;q,t) = n(n - l)/2 , 
deg U(x*; y*; (/, t) = 2kn(n — 1) , 
deg 9, £) = (4k + l)n(n - l)/2 . 

Therefore, the degree of the RHS of (3.1) is less or equal to This concludes the 
proof. □ 

Proof of the theorem 3.1. Again, we can assume 9 = 2k + 1, k = 0, 1, 2, . . . . 

One checks the vanishing of the RHS of (3.1) at the points x = q x , [x (£. A by 
precisely the same argument as in the A-series case, see [Ok3], §4. If x = q x and 
9 is an odd natural number, then the (/-integral is just a finite sum all summand 
of which vanish. Since the denominator D(x*;q,t) does not vanish at x = q x , the 
equality (3.1) holds up to a constant factor. 

To show that this factor equals 1, one can either compute the RHS at x = q 11 
(there will be only one non- vanishing summand in the integral) , or one can consider 
the highest degree term of (3.1). This highest term will be computed explicitly in 
the next subsection. This will conclude the proof of the theorem. □ 

4. Highest degree term 

Denote by 

-fp(*^l? • • • j ^n? 0.1 t) 

the A-series Macdonald polynomial with parameters q and t. 
Theorem 4.1 (Highest degree term). 

(4.1) P*(xi,...,x n ;q,t,s) = P^(xi, x 2 t~ 1 , . . . , x n t 1_n ; q, t) + . . . , 

where dots stand for lower degree terms. 
By definition, set 



n — 7 

X: J 



V(xi,...,x n ) = det 
n(xi,...,a; n ;yi,...,y n _i;g,t) = Jj 



l<i,j <n 



(tyj/xi), 



*,3 

(qxi/x 



D(x 1 ,...,x n ;q,t) = JJ(x 4 - Xj) JJ 

i<j i^j \ 3 ' 

d(n,n) = Y[B q (m + (n-i)6,6), 



J/oo 
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The hats mean that these products are related to the top homogeneous term of 
P*(x;q,t, s). Set also 

Xi — X{t ' , t — 1, . . . , Tl , 

Vi = yit~ l , i = l,...,n-l. 

We shall deduce the theorem 4.1 from the following (/-integral representation 
[Ok3] of P„(x;q,t) 

1 1 f - 

Pf,(x;q,t) = -Tpr- / P^(y;q,t)V(y)U(x;y;q,t)d q y, 

C(ii,n) D(x; q,t) J y ^ x 

where 

t*X\ r^n — i 



\ d q y= ■ ■■ d q y x . . . d q y n -^ 

Jy^x J x? J x„ 



ly^x J x 2 Jx 

Proof . By proposition 2.1 it suffices to consider the = 0. In this case we 

shall use the (/-integral representation of P*(x; (/, t, s) and induction on n. 
By analytic continuation, we can assume that 

t = q e , 9 = 2k+l, k = 0,1,2,... . 

Remark that the highest degree (in variables u and v) term of the polynomial 

equals 

—auv(l — au/v)(l — av/u) = a 2 u 2 (l — av/u)(l — a~ 1 v/u) . 

Therefore 

!!(£*; y*- Q, t) = q k 2 n(n-l)^n-1^2kn(n-l) ^ q "Q ~2fc(n-l) + _ _ _ 

i 

where dots stand for lower degree terms. 
Similarly, 

V( V *) = (r-V2 s) n(n-l)/2 y { ^ + 

i 

and 

D(x*;q,t) = g (2fc+l)fcn(n-l)/2 (t „-l s) (4fc+l)n(n-l)/2 x 

D(x;q,t)l[xf^ + .... 

i 

The powers of (/, t, and s in the three above formulas combine to 

„fc 2 n(n— 1) (+n— 1 „\2kn(n—l) ( +n— 1/2 Nn(n — 1)/2 

9 l ? g J ^ S J _ n(n-l)/4 
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Note that 

dgVi _ dqm_ 
Vi lli 

and 

d -f (...)= f" d -f (...)= f" d -f (...), 

x i+1 yi Jx i+1 yi Jx i+1 yt 

where the first equality is based on (3.2). 

By inductive assumption (note the difference in the definition of Xi and yi) we 
have 

p jl(yu ■ • • » Vn-i; Q, t, s) = t M P^( yi , . . . , y n _i; q,t) + ... . 

Therefore, assuming that the equality (3.1) holds up to a constant factor c, we 
conclude 

Pfj, ("^1 ' ' ' ' ' "^fi> 9? ti C-f/z (£i , . . . , £ n , (/, t) -\- . . . . 

But then setting x n = 1 and using proposition 2.1, inductive hypothesis and the 
stability of A n -series Macdonald polynomials, we immediately find 

c= 1. 

This concludes the proof of the theorem and also the proof of the theorem 3.1 of 
the previous section. □ 

5. Combinatorial formula 
Recall that the polynomials P*(x; q, t, s) form a linear basis in A tjS . 
Definition 5.1. Let the polynomials 

be the coefficients in the following expansion 

(5.1) P*(u, x 2 ,..., x n ; q, t, s) = ^ i>»A u 'i n ) Ki x 2, ...,x n ;q, t, s) . 
Write 

v -< fi 

if A*l > V\ > A*2 > ^2 > ' • ' > V n -1 > Hn- 

Theorem 5.1 (Branching rule). We have 

(5.2) ip^ v (u]n) = 



ut-w n («-«°' (n) t-'' (n) )(i 



1 



? a'(D) t 2n-2-/'(D) s 2 w 



provided v -< fx and ip^ iV (u;n) = otherwise. Here Vv/^ are ^ e sc™ 6 weights that 
appear in the branching rule for ordinary A-type Macdonald polynomials. 

Explicit formulas for the weights ip^/v are given in [M], VI. 6. 24 and Example 
VI.6.2.(b); the last formula is reproduced in (0.4). We will not use explicit formulas 
for ifj^/v in the proof of (5.2). 

The branching rule immediately results in the following combinatorial formula 
for P*(x;q,t,s). Call a tableau T on a diagram fx a reverse tableau if its entries 
strictly decrease down the columns and weakly decrease in the rows. Denote by 
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Theorem 5.2 (Combinatorial formula). We have 
(5.3) P*( Xl , ...,x n ;q,t,s) = J2^U t 1 ^ x 



sen 



(n) t 2(n-T(s))-l'(0) s 2 XT{8) 



where the sum is over all reverse tableaux on \i with entries in {1, . . . , n}. 
Here 

is the same (g, i)- weight of a tableau which enters the combinatorial formula for 
ordinary Macdonald polynomials (see [M],§VI.7) 

Ptx(x; q,t) = JJ x T{s) . 

Comparing this combinatorial formula with the combinatorial formula [Ok3] for 
the 2-parametric interpolation Macdonald polynomials P*(x;q,t) one obtains the 
following proposition, in which part (b) follows from (2.2). 

Proposition 5.3. 

a) P*(x;q,t,s) -> P*(x;q,t) , s -> oo , 

b) s- 2 ^lp;(x;Q,t,s) ^t( 2 - 2n )l^lp;(l/a:;l/Q,l/t), s^O. 

In the proof of the branching rule we shall induct on n and use the following 
corollary of this theorem 

Corollary 5.4 of theorem 5.1. For all r = 1, . . . ,n we have 

(5.4) P*(x u ...,x r , q^+\. ..,q^;q, t, s) = cx^ . . . x^ + . . . , 

where c is a non-zero constant and dots stand for lower monomials in lexicographic 
order. 

Proof. Given any partition v set 



Set also 



Consider the leading term of P*(x;q,t,s) as of a polynomial in x\. Then (5.2) 
asserts that this leading term equals 

(5.5) xf t-I^P^(x 2 ,...,x n ;g,t,s), 

because = Using (5.2) again one obtains the leading term of (5.5) in x 2 

and so on. Finally, observe that 



pronto 



for alH. □ 
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Lemma 5.5. Let X be a partition. Then 

P*(x 1 ,...,x i _ 1 ,q Xi ,...,q Xn ;q,t,s) = Q unless C W A . 

Proof. By definition of P*(x; q, t, s), this polynomial vanishes at all points 

X! =q 1 ,...,x i -i = q 

where 

!/!>•••> > A; 

are arbitrary integers, provided ^ W A. □ 

Proof of theorem 5.1. Induct on n. The case n = 1 is clear. 
Fix some z. Show that if Vi < [ii then 

(5.6) ^(u; n) = , u = ^t 1 " 1 , . . . , ^"V"* . 

We shall prove (5.6) by induction on the partition <%, that is we shall deduce (5.6) 
from the assumption that 

W,>;n)=0, u = q^t 1 -\...,q^-H 1 -\ 

for all 7] such that 
Suppose 

u = q k t 1 ~\ k = u i ,...,fji i -l, 
and consider the expansion 

(5.7) P; (^t 1 " 1 , x 2 , . . . , Xi , q»% . . . , q Vn ~ x \ q, t, s) = 



= ^W?** 1 ^ n ) P ^( x 2, • • • , Xi, q Vi , . . . , q^" 1 ; g, £, s) . 



v 

By the lemma only summands satisfying 



are nonzero. On the other hand, if 



then in particular 77^ < z/j and by our assumption ift^^qH 1 = 0. Therefore 
only summands with 

«) v = ® u 

enter the sum. 

By the corollary 5.4 applied to polynomial 
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in n — 1 variable, each summand in (5.7) has the following form 
(5.8) c r) ^, v (q k t 1 - i ;n){xf...x 7 l i - 1 +...), 

where c v is a nonzero factor and dots stand for lower monomials in lexicographic 
order. 

On the other hand, by the symmetry of P^(x; q, t, s) we have 
PZtft 1 -*, x 2 , . . . , Xi , q Ui , . . . , g"- 1 ; q, t, s) = 

= P;(x 2 /t, . . . , Xi/t, q\q v \..., g""" 1 ; ?, M) = 

by lemma 5.5, provided Ui < k < fii. Therefore the polynomial (5.7) is identically 
zero. By virtue of (5.8) it is impossible unless 

^, v (q k t 1 -*;n) = 

for all r) such that (i) r] = (i V. This proves (5.6). 

Since ip^ jU (u;n) is invariant under the transformation 

1 



u 



t 2n 2 S 2 U 

it vanishes also at the points 

1 



t 2n-2 s 2 u -<l '•••'9 

for all i such that i/j < 
Now show that 

ip,j,A u 'i n ) = 

\iv (]L \i. Suppose that 

i/ £ /x 

and ip^fa, n) is not identically zero. Since we know some zeroes of the polynomial 

ipf Jl ,i/(u;n) we have 

deg^ M ,i/(u;n) > ^max{/ii - 0} . 

i 

Hence _ 

degip^u^n) P*(x 2 , . . . ,x n ;q,t, s) > ^max^,^} > \fi\ . 

i 

Therefore such a summand cannot occur in the expansion (5.1). 
Thus we can assume that 

v C fj, . 

Since we know 2\/j,/u\ distinct zeroes of ^ M)l/ (tt; n) and again 

degVv^(>; n ) < l/VH 
this polynomial should up to a scalar factor equal 

TT ( u _ V (□)£-/' (crA f x 1 A 

11 1 ^ J\ „a'{U) t 2n-2-V{U) s 2 u ■ 

This factor equals Vv/^ ^ because the highest degree term of 

P^ (x\ , . . . , x n ; (/, £, s) 
is the A-series Macdonald polynomial 

Pfi(xi, x 2 t~ 1 , x n t x ~ n ; q, t) . 

TVii« rnnnliirl(='« Itip nrnnf 
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6. PlERI FORMULAS AND CAUCHY IDENTITY 

Definition 6.1. Let ifj' x ^(u, n) be the coefficients in the following expansion 

(6.1) f[(u + x^) (l + -^_L_^ P ;( X - q , t , s) = 

i=l ^ 1 ' 

A 

It is clear that ifj' x ^(u,n) is a polynomial in u and n _1 invariant under the 
transformation 

1 

u h-> „„ _„ . 
s t u 

Denote by \i + 1 the following partition 

fJL + l = + + 

Denote by ipx/^ the coefficients of the Pieri formula for ordinary A n -type Mac- 
donald polynomials (see [M], VI. 6) 

n 

(6.2) H (u + Xi ) P^x; q,t) = J2 A/» ^ +VM Q, t) . 
i=l A 

Theorem 6.1 (Pieri formulas). We have 
(6-3) ip' x ^{u,n) = 

n ( M+ /' a ' t -'-' a »)(i + sV , (0)(2( U-,(o, M ). 

□e/i+i/A \ y / 

and V'a ^l -1 ^ n) = otherwise. 

Pieri formulas for the interpolation Jack polynomials were considered in [KS] . 
Proof. First show that 

(6.4) ^A,>,n) = if ^£A 

by induction on A. Assume that 

for all partitions n such that n ^ A . Then evaluation of (6.1) at x = q x gives (6.4). 
From now on we suppose 
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Set h'q = n and suppose that X' k < f° r some k > 1. Show that 
(6.5) iP' x J-q k -H 1 -\n)=0, 1 = ^ + 1,...,^. 

Again, we assume that 

^J-Q^t 1 ' 1 , n) = , i = rj' k + l,..., fi' k+1 . 
for all partitions rj such that 

and evaluate (6.1) at 

x = q\ u = -q k - 1 t 1 -\ i = A / fc + l,...,At' fc+ i. 

This gives (6.5). 

Since ^(^n) is invariant with respect to the transformation 

1 

s 2 t 2n ~ 2 u ' 

we have 

( 6 - 6 ) ^A, M (- ^gfc-l^n-i-l ' n ) = > » = A' fc + 1, . . . , //' fc+1 • 

Now prove that 
(6.7) ^>,n) = if 

Suppose that ^ ^(w, n) 7^ 0. Then by (6.5) and (6.6) 

If A <(_ n + 1 then 

i 

but then such a summand cannot enter the RHS of (6.1). 
If A C \i + 1 then by the same reason 

degV^Cu,™) < -h 1/Aj . 

By (6.5) and (6.6) we have 

const n (« + f' a '«-^») (l + (a)f2( U-„ (0); , ) ■ 

The constant factor is determined by (4.1) and (6.2). This concludes the proof of 
the theorem. □ 

The similarity of the proof of Pieri formulas and of the combinatorial formula is 
not accidental. In fact, these two theorems are equivalent to each other by virtue 
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Theorem 6.2 (Cauchy identity). 

( 6 . 8 ) ii n - (i - s ^- H „-, XWi ) 



= (-i) l ^ | t (n - 1)H 9 (m - 1)l ' I| J p;(^9,t^)^(2/;t,9^), 

/icK) 

where 

It follows from (6.8) that the branching rule in variables x is equivalent to Pieri 
formula in variables y and vice versa. 

Lemma 6.3. Suppose [i C (m n ) and A has at most m parts. Then 

n m 

(6.9) Y[ Y[ (q^t n ~ l - t x iq m - J ) = 

unless jl C A. 

Proof of the lemma. Suppose this product does not vanish. Then 

A m - Mi ^n-i, i = 1, ... ,n. 
For % = fi[ + 1, . . . , n we obtain 

X m ^ 0, 1, . . .,71 - /iti - 1 . 

Therefore 

A m > ra - /ii . 

For % = n' 2 + 1, . . • , fi[ we obtain 

A m _i 7^ n - //i, . . . , n - /j' 2 - 1 . 
Since X m -i > A m > n — n[ we obtain 

A m _i > n - ^2 • 

In the same way we obtain 

A m _ 2 > n - /4, • • • , Ai > n - /i' m , 

which means Ji C A. □ 

Proof of the theorem. Denote by f(x; y) the product in the LHS of (6.8). Observe 
that 

f(x;y) e A t:S [x] and f(x; y) E A qi8 [y] . 
Expand f(x; y) in polynomials P^(x; q, t, s) 

f{x;y) = Y J P;{^^s)Pl{y) 
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where P^(y) are certain elements of A g>s [y] of degree 

degP^(y) <\J1\. 

Since m is the highest exponent of x\ in f(x;y) only partitions [i satisfying 

At C (m n ) 

enter the expansion of f(x; y). We have to prove that 
(6.10) Pl{y) = (-l)fck( n -VMq( m -VM PZ(y-,t,q t s) . 

Induct on Assume that 

P, ? (y) = (-lf\ t (n-l)M q (rn-l)\v\ p£( y . q , s ) , 

for all partitions r\ such that 



Then we have 



Let A range over all diagrams with < \J1\ squares. Then 

PI(t A ;t,<Z, s) = 

because \rj\ > By the lemma 

/(^;t A ) = 

unless A = /L Therefore 

Pl(t x ) = 

unless A = Ji. This proves (6.10) up to a scalar factor. This factor is determined by 
(4.1) and the following formula (6.11) for ordinary Macdonald polynomials which 
we recall. □ 

Proposition 6.4. 

n m 

(6.11) nn^-^)= E (-^) M P t ,(x;q,t)P~(y,t,q). 

i=lj=l nC(m n ) 

Proof. The identity (5.4) in [M], Ch. VI, reads 

n m 

+ Xiyj) = E P^;q,t)P^(y,t,q). 

i=lj=l fJ,C(m n ) 

Therefore (6.11) is equivalent to 



Pfo;t,Q)= (jly^jP^(i/y;t,q). 



It is clear that the RHS of the above equality is a polynomial in y. One easily checks 
that it is an eigenfunction of the Macdonald (/-difference operator D\ defined in 
[Ml formula. (VT.3.4Y l~l 
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7. Binomial formula for Koornwinder polynomials 
The Koornwinder polynomials [K] 

P\(x; q, t, ai, . . . , a 4 ) G C(g, £, a u . . . , a^xf 1 , . . . , X* 1 ] 

are VF-invariant and depend on six parameters 

q,t,a 1 ,a 2 ,a 3 ,a 4 . 

These polynomials are orthogonal on the torus 

X o — 1 • % — 1 « • • • « T\i 



with respect to following measure 



1 . , . dx 
- - A(x) — , 



where 



A ( a; ) = ^ 1 ±1 3 ±1 j °° 

(tX i Xa )oc 



n 



(A 



±1 _ x ±1 a 1 / 2 ^ 1 _«V2^±1 



(/ ' X 



q ' x's 



(axxf 1 , -a 2 xf\ q 1/2 a 3 xf\ -q 1/2 a^xf l ) 



±i 



i<j 

Here, by definition, 

(it, V, . . . , 10)00 = (m)oo(^)oo • • • («>)oo 

and 

(it) 00 = (1 - u)(l - git)(l - g 2 -u) 

These polynomials specialize to Macdonald polynomials for classical root systems, 
see [K] and also [Dl], section 5. It is known ([Dl], section 5.2) that the highest 
degree term of P\ is the A-type Macdonald polynomial with parameters q and t. 

The parameters ai,...,a4 are related to parameters a,b,c,d used by Koorn- 
winder via 

a = a±, b=—(i2, c = q 1/2 as, d=—q 1/2 ci4 : . 

It is also convenient to introduce, following J. F. van Diejen, dual parameters 
a[, . . . , a' 4 related to parameters a±, . . . , 0,4 by the following involution 

/ log a[ \ 

log a' 2 

log a' 3 
V log a' A I 





f 1 


1 1 






/log 


;ai\ 


1 


1 


1 -1 


-1 




log 




2 


1 


-1 1 


-1 




log 


;a 3 




Vi 


-1 -1 


1/ 




Vlog 


;a4 / 



In particular, 



To simplify notation, we shall sometimes omit the six parameters and write simply 
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A (/-difference operator whose eigenfunctions are P\(x; q, t, a±, . . . , 04) was found 
by T. Koornwinder in [K]. In [Dl] van Diejen found explicit (/-difference operators 
Dfc, k = 1, . . . , n such that 

D k P x = E k (q x )P x 

where 

and the highest degree term of E k is (up to a constant factor) the k-th. elementary 
symmetric function in variables q Xi t n ~' 1 . 

The operators Di, . . . , D n have the following crucial property. Define the vectors 
p and p' by 

q p = (t n - 1 a' 1 ,...,ta[,a' 1 ), 
qp' = (t n - 1 a u ...,ta u a 1 ). 

Suppose A is a partition and consider the number 
(7-1) [D k P\] (q^ +p ') . 

Since D k is a (/-difference operator this number is a combination of values of P\ 
at several neighboring points with some coefficients which do not depend on A. In 
fact, see Lemma 4.3 in [D2], only points of the form 

x = q»+o' , 

where v is a partition and 

ki-A*i|<l, ^ \v% - ^i\<k 
contribute to (7.1). Moreover, the contribution of the point 

v = (^1 + 1, . . . , p k + 1, p k+u ...,n n ) 

is nonzero. 

In the context of Cherednik's double affine Hecke algebra such a property of in- 
difference operators can be proved in an abstract fashion, see [CI, 2]. One can apply 
the Cherednik's algebra techniques to Koornwinder polynomials, see [M2,Nol]. 

This property results in the following theorem 

Theorem 7.1 (Binomial formula). 

P\(xq p ';q,t 1 a 1 ,... 1 a4) _ 
P\{q p ']q,t,a 1 ,...,a A ) 

Y^+(n-i)lML t 

1 P^(q^;q,t,a[)P^(qP' ';g,*,ai, . . . ,a 4 ) ' 

Recall that an explicit product expression for P*((/ M ; (/, t, a[) is given in definition 
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Proof. In the same way as in the A-series case (see the proof of the main theorem 
in [Ok4]) it follows from the above properties of the operators Dk that for any 
partition fi there exists an (/-difference operator such that 

[D^P X ] (/) = Pa(9 m+p '). 
This operator D M is a polynomial in Di, . . . , D n and 

where 



and 



df, e A t>ai 



degc^ < 



Consider the Newton interpolation of the function 

= P\{xq p ';q,t,a 1 ,...,a4) 

P\{q p ';q,t,ai,---,a4) 

with knots 

x = q^ , 

where \i ranges over partitions with at most n parts. This Newton interpolation 
has the form 

f{?) = b^^xP* (x; q, t, ai) , 

A* 

where is a linear combination of the values f(q u ) for u C fx. We have 

/(0 = d.(« A ). 



Therefore, 

for some polynomial 
of degree 



b^,x = b^(q x ) 

V e A t , a; 



deg^ < . 

Since the highest degree term of f(x) equals 



t (n-l)|A|JA| 

-P x (x l ,x 2 t 1 ,...,x n t 1 n ;q,t), 



P A (Q p ';Q,t,ai,...,a 4 )' 



where Px(x;q,t) is the ordinary A-type Macdonald polynomial, and the highest 
degree term of P*{x; q, t, a\) equals 
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we have 

A 



W) = 



0, |A|<H,A^/x, 
t (n-i)IH a H/P M (/ ; t, ai , . . . , a 4 ), A = fi . 



Since deg5 M < | /x J the polynomial 6 M is completely determined by its values at the 
points g A , where |A| < \fx\. Therefore b^(q x ) is proportional to P^(q x ; q, t, a[) and 
precisely 

( X)= ^-DHqH P;(g A ;g,t,aj) 
AQ ' P M (^';9,t,a 1 ,...,a 4 )P;(^;9,t,a / 1 )" 

This concludes the proof. □ 

The following important property of Koornwinder polynomials immediately fol- 
lows from the binomial theorem 

Theorem (J. P. van Diejen, [D2]). If a[ = a\ then 

P\{q u+P ';q,t,a 1 ,...,a 4 ) = Py{q X+p ; q,t, a u . . . , a 4 ) 
P x (qP';q,t,a 1 ,...,a 4 ) P v {q p ; q,t, a u . . . , a 4 ) 

The general symmetry, 

Px(q u+P ';q, t, ai, . . . , a 4 ) P„{q x+P ; q, t, a[, ...,<) 



(7.2) 



P x (qp';q, t, a u . . . , a 4 ) ^(^; 9, *, ai, ...,<) 



conjectured in [D2], depends on a formula for P\(q p ;q,t,ai,..., a 4 ). The conjec- 
tural formula for this number (see formula (5.5) in [D2]) was proved in [D2] under 
the self-duality condition 

a' x = cii . 

According to the note added in proof to [D2], that formula (together with sym- 
metry (7.2)) was proved recently by I. G. Macdonald (in preparation). 

Appendix 1. ^-Integrals 
Recall that (see, for example, Ch. 1 of [GR]) 

(A.l) f U z ld f = J L {u i_ v i ) , ^o, [l] q = 1 -^-- 

Jv z [ l \q 1 ~q 

If u = q v for some k G N then 



ru j k 1 

J V Z n 



for any polynomial f(z) without constant term. For example, polynomials f(z) G 
C[z ±:L ] which satisfy 



26 ANDREI OKOUNKOV 

do not have constant terms. Suppose an analytic function f(z) is given by its 
Laurent series of the form 

oo 

f( Z ) = J2cn(z n -Z- n ), S-'k^KS, 
1 

for some S > 1. By definition, set 

/" /w feHi-,)E-(iV-r^i 



oo 



z=v 

z=u 



( A -2) = (1 - 9) E q - n/ 2 Z q n/2 ((^" 1/2 ) n + (^- 1/2 )" n ) 

which converges if 

|<?|5 _1 < \v\, \u\ < 6 . 
Note the following symmetries of (A. 2) 

r / w ^ = r /u / w ^ = r / w ^ = - r / ^ ^ . 

Recall that the g-beta function is defined (see [GR],1.10) by 

Appendix 2. Absence of rational ^-difference equations 
In this Appendix we show that the following polynomials f(x) in one variable x 

fm(%) = Pm( x 'i (?) tj s ) 

= (x-l)(x- q )...(x- q ™- 1 )(l-±-).--(l- ' 



s 2 x J \ s 2 q m 1 x 
where 

m = 1, 2, . . . , 
do not satisfy any g-difference equation of the form 

(A.4) a i (x)f m (q i x) = E(m)f m (x), d>0, 

-d<i<d 

where cii(x) are rational functions in x and 

a d (x) ^ or a-d(x) ^ 0. 
Assume the polynomials f m {x) satisfy (A.4). Consider the points 
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Only finitely many of them are poles of functions a,i(x), —d < i < d. Evaluate 
(A. 4) at the remaining points. We obtain 

a d (q m - d )f m (q m ) =0. 

Since f m {q m ) ^ we obtain 

a d (q m - d )=0 
for infinitely many values of m. Therefore 

a d (x) = 0. 

Similarly, evaluation at the points 

x = -j, m = 2d, 2d + 1, . . . 

gives 

a-d ( — t I = 

\s 2 q m - d J 

for infinitely many values of m. Therefore 

a- d (x) = . 
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